ThreadSanitizer: lock-order-inversion (potential deadlock) [@ mozilla::detail::MutexImpl::lock] through [@ ensureRegisteredWithProfiler]
Categories
(Core :: JavaScript Engine, defect, P1)
Tracking
()
People
(Reporter: decoder, Assigned: nbp)
References
(Blocks 1 open bug, Regression)
Details
(Keywords: regression)
Attachments
(2 files)
The attached crash information was detected while running CI tests with ThreadSanitizer on mozilla-central revision c0a6eb95b65c.
This appears to be a potential deadlock with profiling canceling an off-thread compilation, causing us to unlock things in the wrong order.
General information about TSan reports
Why fix races?
Data races are undefined behavior and can cause crashes as well as correctness issues. Compiler optimizations can cause racy code to have unpredictable and hard-to-reproduce behavior.
Rating
If you think this race can cause crashes or correctness issues, it would be great to rate the bug appropriately as P1/P2 and/or indicating this in the bug. This makes it a lot easier for us to assess the actual impact that these reports make and if they are helpful to you.
False Positives / Benign Races
Typically, races reported by TSan are not false positives [1], but it is possible that the race is benign. Even in this case it would be nice to come up with a fix if it is easily doable and does not regress performance. Every race that we cannot fix will have to remain on the suppression list and slows down the overall TSan performance. Also note that seemingly benign races can possibly be harmful (also depending on the compiler, optimizations and the architecture) [2][3].
[1] One major exception is the involvement of uninstrumented code from third-party libraries.
[2] http://software.intel.com/en-us/blogs/2013/01/06/benign-data-races-what-could-possibly-go-wrong
[3] How to miscompile programs with "benign" data races: https://www.usenix.org/legacy/events/hotpar11/tech/final_files/Boehm.pdf
Suppressing unfixable races
If the bug cannot be fixed, then a runtime suppression needs to be added in mozglue/build/TsanOptions.cpp. The suppressions match on the full stack, so it should be picked such that it is unique to this particular race. The bug number of this bug should also be included so we have some documentation on why this suppression was added.
|
Reporter | |
Comment 1•5 years ago
|
||
|
Assignee | |
Comment 2•5 years ago
|
||
The issue seems to be that when we start the profiler, we first take the profiler lock and then cancel all compilation take the helper thread lock for JS helper threads.
However, when we create a new JS Helper thread we take the helper thread lock to access the registered callbacks, and then take the profiler lock to register the threads.
Apparently taking the helper thread lock might be unnecessary as the callbacks are WriteOnce fields. Therefore I suggest we move profiler function calls out-side the helper thread locked section.
I will write a simple patch to fix this.
|
||
Updated•5 years ago
|
|
|
Assignee | |
Comment 3•5 years ago
|
||
|
||
Updated•5 years ago
|
|
||
Comment 5•5 years ago
|
||
| bugherder | ||
|
||
Comment 6•5 years ago
|
||
I assume we don't need to uplift this to Beta, but feel free to nominate for approval if you feel strongly otherwise.
|
|
Assignee | |
Comment 7•5 years ago
|
||
I do not have a good sense of the importance of this bug in terms of hangs while using the profiler.
Greg, any idea if we should backport this fix?
|
||
Comment 8•5 years ago
|
||
I'm not sure I've encountered this personally, so I don't have a big opinion on uplifting here. I'll ping :gerald though to see what he thinks.
Sorry, the js part of the profiler is still opaque to me. I also haven't experienced this issue, so it doesn't seem that critical.
Kicking the can towards Markus...
|
|
||
Updated•5 years ago
|
Description
•